Skip to content

feat: disable PeriodicTasks removed from beat_schedule#1035

Open
serl wants to merge 7 commits into
celery:mainfrom
serl:feat/from-configuration-flag
Open

feat: disable PeriodicTasks removed from beat_schedule#1035
serl wants to merge 7 commits into
celery:mainfrom
serl:feat/from-configuration-flag

Conversation

@serl
Copy link
Copy Markdown

@serl serl commented Apr 30, 2026

Track tasks imported from CELERY_BEAT_SCHEDULE with a new from_configuration boolean on PeriodicTask.

On beat startup, rows flagged from_configuration=True whose name is no longer in the config are disabled (not deleted) so history is preserved and admin/ORM created tasks are never touched.
Those tasks are re-enabled when re-added to the configuration.

NOTE: This changes the behavior, as the enabled flag was not overridden on worker restart before. Specifically, if a task is in CELERY_BEAT_SCHEDULE and manually disabled in the admin, this would re-enable it. A way out would be to hard delete the deleted tasks (and stop manipulating enabled)

On the other hand, this PR also shows the flag in the admin and warns editors that changes to imported tasks are reverted on the next beat restart.

Closes #248, #654.

Linked to #1034, IMO both are interesting to have, but no hard feelings either way (or no way 😁)

serl and others added 2 commits April 30, 2026 15:45
Track tasks imported from CELERY_BEAT_SCHEDULE with a new
from_configuration boolean on PeriodicTask. On beat startup, rows
flagged from_configuration=True whose name is no longer in the config
are disabled (not deleted) so history is preserved and admin/ORM
created tasks are never touched. Surface the flag in the admin and
warn editors that changes to imported tasks are reverted on the next
beat restart.

Closes celery#248, celery#654.

Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>
Make CELERY_BEAT_SCHEDULE the source of truth for the enabled flag on
imported tasks: removing-then-re-adding a task to the configuration
re-enables the existing row, instead of leaving it stuck in the
auto-disabled state from the previous orphan-cleanup pass. The admin
warning banner already advertises this revert-on-restart behavior.

Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>
Copy link
Copy Markdown

Copilot AI left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pull request overview

Adds explicit tracking for periodic tasks imported from CELERY_BEAT_SCHEDULE and changes scheduler startup behavior to disable (not delete) DB rows that are no longer present in configuration, preserving history and avoiding impacting admin/ORM-created tasks.

Changes:

  • Add PeriodicTask.from_configuration boolean field (with migration) and mark imported tasks accordingly.
  • On beat startup, disable from_configuration=True tasks missing from current configuration and re-enable when re-added.
  • Expose the flag in Django admin and show a warning on the change form; expand unit tests and docs to reflect the new behavior.

Reviewed changes

Copilot reviewed 6 out of 6 changed files in this pull request and generated 3 comments.

Show a summary per file
File Description
django_celery_beat/schedulers.py Marks config-imported rows, forces enabled=True for config entries, and disables orphaned imported tasks on startup.
django_celery_beat/models.py Introduces from_configuration field on PeriodicTask.
django_celery_beat/migrations/0020_periodictask_from_configuration.py Adds the DB migration for from_configuration.
django_celery_beat/admin.py Shows from_configuration in admin and warns when editing imported tasks.
t/unit/test_schedulers.py Updates/extends tests for re-enable behavior, orphan disabling, and admin warning message.
docs/includes/introduction.txt Documents how imported tasks behave on restart and how admin edits are handled.

💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.

Comment thread django_celery_beat/schedulers.py
Comment thread django_celery_beat/schedulers.py
Comment thread t/unit/test_schedulers.py Outdated
auvipy and others added 5 commits May 13, 2026 14:56
Co-authored-by: Copilot Autofix powered by AI <175728472+Copilot@users.noreply.github.com>
…attern

Add a regression test for the case where a previously valid beat_schedule
entry is edited into an invalid form: the existing DB row must be disabled,
not left running on the now-stale schedule.

While here, refactor the surrounding tests in the class to instantiate the
scheduler once and call setup_schedule() to reprocess config changes,
rather than re-instantiating the Scheduler each time.

Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>
The trailing ``self._schedule = {}`` / ``self._heap_invalidated = True``
in ``setup_schedule`` intended to force a rebuild after
``_disable_removed_from_configuration``, but it left the scheduler in a
broken state.

By the time those lines ran, the earlier ``self.schedule`` access from
``install_default_entries`` had already flipped ``_initial_read`` to
False. Clearing ``_schedule`` alone is not a rebuild trigger -- on the
next access the property goes through the ``elif schedule_changed()``
branch, which returns False (``_last_timestamp`` was just set to the
current ts, so ``ts > ts`` is False), so the empty dict was returned.
That made 10 tests in ``test_DatabaseScheduler`` /
``test_DatabaseSchedulerFromAppConf`` fail with ``KeyError`` /
``assert {}`` on what should be a populated schedule.

Invalidation is already handled correctly:
``_disable_removed_from_configuration`` calls
``PeriodicTasks.update_changed()`` whenever it disables something,
which bumps the change timestamp and makes the next
``schedule_changed()`` return True -- triggering the normal rebuild
path that also clears the heap.

Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>
@serl
Copy link
Copy Markdown
Author

serl commented May 15, 2026

NOTE: This changes the behavior, as the enabled flag was not overridden on worker restart before. Specifically, if a task is in CELERY_BEAT_SCHEDULE and manually disabled in the admin, this would re-enable it. A way out would be to hard delete the deleted tasks (and stop manipulating enabled)

@auvipy actually I have a doubt about this point. Maybe it's better to remove the tasks from database instead of touching the enabled flag? Which one do you think is more correct and/or less confusing for users?

@codecov
Copy link
Copy Markdown

codecov Bot commented May 16, 2026

Codecov Report

❌ Patch coverage is 96.42857% with 1 line in your changes missing coverage. Please review.
✅ Project coverage is 88.73%. Comparing base (99f7bcf) to head (f611c51).
⚠️ Report is 2 commits behind head on main.

Files with missing lines Patch % Lines
django_celery_beat/admin.py 80.00% 0 Missing and 1 partial ⚠️
Additional details and impacted files
@@            Coverage Diff             @@
##             main    #1035      +/-   ##
==========================================
+ Coverage   87.64%   88.73%   +1.09%     
==========================================
  Files          32       33       +1     
  Lines        1012     1039      +27     
  Branches       81       84       +3     
==========================================
+ Hits          887      922      +35     
+ Misses        107       97      -10     
- Partials       18       20       +2     

☔ View full report in Codecov by Sentry.
📢 Have feedback on the report? Share it here.

🚀 New features to boost your workflow:
  • ❄️ Test Analytics: Detect flaky tests, report on failures, and find test suite problems.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

Removing tasks from celery_beat config doesn't remove them from database.

3 participants